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LONG-TERM  GOALS 

The  goal  of  this  project  is  to  develop  and  implement  a  comprehensive  data  management  and 
distribution  strategy  that  allows  easy  and  efficient  access  to  HYCOM-based  ocean  prediction  system 
outputs  to  (a)  coastal  and  regional  modeling  sites,  (b)  to  the  wider  oceanographic  and  scientific 
community  including  climate  and  ecosystem  researchers,  and  (c)  the  general  public  especially  students 
in  middle  and  high  schools. 

OBJECTIVES 

To  develop  and  implement  a  data  sharing  system  using  existing  open  source  software  packages  to 
distribute  HYCOM-based  ocean  prediction  system  data  via  the  Internet. 

APPROACH 

The  basic  idea  is  to  setup  web  services  that  enable  access  to  backend  data  management,  distribution 
and  visualization  applications.  These  applications  enable  end  users  to  obtain  a  broad  range  of  services 
such  as  browsing  of  datasets,  gif  images,  downloading  NetCDF  files,  FTP  request  of  data  etc.  The 
HYCOM  Data  Sharing  System  is  built  upon  three  existing  software  components:  the  Open  Project  for 
Network  Data  Access  Protocol  (OPeNDAP)  Servers,  Unidata’s  Thematic  Realtime  Environmental 
Distributed  Data  Services  (THREDDS)  server,  and  the  Live  Access  Server  (LAS).  These  tools  and 
their  use  to  distribute  the  data  are  described  below. 

a)  OPeNDAP 

OPeNDAP  (http://www.opendap.org)  is  the  middleware  that  provides  uniform  binary-level  access  to 
scientific  data  on  the  Internet.  There  are  three  components  to  OPeNDAP:  servers,  client  libraries,  and 
a  format-neutral  data  transport  protocol.  The  protocol  may  be  thought  of  as  XML-tagged  data 
descriptions  (metadata)  with  associated  non-parsed  (binary  data)  content.  OPeNDAP  servers  provide 
access  to  existing  formatted  files  (e.g.,  netCDF,  HDF  etc.)  through  the  OPeNDAP  Data  Access 
Protocol.  Servers  are  typically  available  as  binary  executables  that  the  data  provider  can  quickly  install 
Existing  and  new  client  applications  are  supported  through  the  OPeNDAP  client  libraries.  Support  for 
clients  that  utilize  netCDF  is  particularly  simple,  as  the  netCDF  native  libraries  have  been  OPeNDAP- 
enabled  so  that  a  simple  relinking  of  an  application  is  often  sufficient  to  OPeNDAP-enable  it  (see 
Figure  1). 


1 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

30  SEP  2007  2' REPORT  TYPE 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

HYCOM  Data  Service  And  Web  Outreach 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Florida  State  University, Center  for  Ocean-Atmospheric  Prediction 

Studies, Tallahassee, FL, 32306 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited. 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

/\PlcITT?  act 

1 8 .  NUMBER  1 9a.  NAME  OF 

rtU  D  A  CPC  PtnQPrtMQTPT  U  DUDCHM 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

unclassified  unclassified  unclassified 

10 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


A) 


B) 


Figure  1.  Adding  OPeNDAP  capabilities  to  an  existing  application.  Figure  A  illustrates 
the  Application  Programmer  Interface  (API)  as  the  boundary  across  which  a  scientific 
application  makes  requests  to  its  local  file  access  libraries  (e.g.,  netCDF)  to  read  and 
write  data  files.  Figure  B  illustrates  the  same  scientific  application  relinked  with 
OPeNDAP.  The  OPeNDAP  client  mimics  the  API  used  for  local  file  access,  directing 
the  requests  to  the  local  file  access  libraries,  or  to  the  network  as  appropriate. 


The  OPeNDAP  group  has  also  developed  a  HYCOM  specific  data  access  GUI  tool  for  use  with 
MATLAB  (see  Figure  2  below).  It  is  available  form  www.opendap.org  . 


O  O  O  CULHYCOM 


Figure  2.  Screen  shot  of  the  MATLAB  GUI  OPeNDAP  client  developed 

for  HYCOM  data  access. 
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b)  THREDDS  Data  Server 

The  THREDDS  Data  Server  (TDS)  is  a  web  server  that  provides  cataloging  service,  metadata  and  data 
access  for  scientific  datasets.  It  consists  of  an  integrated  server  that  provides  OPeNDAP  access  to  any 
dataset  stored  in  a  format  compatible  with  the  Common  Data  Format  such  as  NetCDF  etc.  The  server 
also  provides  bulk  file  access  through  the  HTTP  protocol  and  data  access  through  the  OpenGIS 
Consortium  (OGC)  Web  Coverage  Service  (WCS)  protocol  for  any  "gridded"  dataset  whose 
coordinate  system  information  is  complete.  It  also  allows  netcdf  files  to  be  aggregated  to  allow  access 
to  data  stored  in  multiple  files  as  a  single  logical  file.  The  HYCOM  THREDDS  catalog  is  accessible  at 
http://hycom.coaps.fsu.edu/thredds/dodsC/  and  is  shown  below. 


ICatalog  http://hycom.coaps.fsu.edu/thredds/dodsC/catalog.xml 
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Dataset 

Size 

Last  Modified 

^  HYCOM  1/12  Global  Datasets  -  (Test) 

-  1 

Global  Non-Assimilative  Simulation  (2003-2005)/ 

- 

□  Global  NCODA  Analysis  (2003-2004)/ 

-  1 

□  Global  NCODA  Analysis  (April-2007  -  Present)/ 

--  1 

Atlantic  Ocean  Prediction  System  Products  (in  native  coordinates) 

- 

Best-Estimate  Time  Series  (2003-Present)/ 

- 

Weekly  Forecast/ 

--  1 

Weekly  Hindcast/ 

—  1 

Atlantic  Ocean  Prediction  System  Products  (Diagnostic  Product  at  Levitus  Depths 

—  1 

North  Atlantic  Best-Estimate/ 

-- 

CD  North  Atlantic  MERSEA  Classl  Best-Estimate/ 

""  1 

North  Atlantic  MERSEA  Classl  Latest  Bulletin/ 

-  1 

North  Atlantic  MERSEA  Classl  Forecast  Bulletin/ 

- 

North  Atlantic  MERSEA  Classl  Hindcast  Bulletin/ 

-  1 

Pacific-Interannual-ECMWF  Simulation  (1979-2003) 

-  1 

^  Pacific-Interannual-ECMWF  Simulation  (1979-2003)/ 

- 

^  COAPS  -  QSCAT  WINDS  (1999-Present) 

-- 

Q  QSCAT-OO/ 

-  1 

Q  QSCAT-06/ 

—  1 

LJ  QSCAT-12/ 

- 

Q  QSCAT-18/ 

- 

Internal  Data  Sets 

— 

Control/ 

- 

CD  MVOI/ 

-- 

CD  ENOI/ 

- 

Rl 

2d 

http://hycom.coaps.fsu.edu/thredds/dodsC/Hindcast/atl_ops_hindcast_catalog.html 

1  1  1  1  1  IIO  We™* 

\  100%  ▼  J 

Figure  3.  Screenshot  of  the  HYCOM  THREDDS  catalog  listing  the  available  datasets. 


c)  LAS 

The  FAS  (http://www.ferret.noaa.gov/FAS/)  is  a  configurable  scientific  data  "product"  server  (see 
Figure  2).  It  provides  a  friendly  user  interface  to  ocean/atmosphere/climate  data  browsing,  download, 
and  comparison  on  the  Web.  A  user  can  quickly  obtain  products  such  as  plots,  images,  and  formatted 
files  generated  on-the-fly  from  custom  subsets  of  variables. 
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Figure  4.  The  heart  of  LAS  is  a  "product  server " 


During  FY07  major  enhancements  have  been  made  to  LAS  in  support  of  HYCOM  modeling.  These 
include  the  LAS  “Slide  Sorter”;  animations;  outputs  to  Google  Earth;  GIS  compatibility;  and 
improvements  to  the  LAS  code  framework.  These  will  be  discussed  separately,  below: 

i)  Slide_Sorter 

Model  intercomparison,  and  indeed  analysis  of  single  models,  is  challenging  in  part  because  the 
number  of  fields  that  need  be  examined  is  so  large.  The  LAS  Slide  Sorter  is  a  specialized  interface  to 
assist  HYCOM  scientists  with  this  challenge.  If  we  think  of  a  HYCOM  model  ensemble  as  occupying 
a  5-dimensional  space  (3-space,  time,  ensemble  axis),  then  a  2D  contour  plot  leaves  three  axes  to 
navigate.  The  Slide  Sorter  allows  the  user  to  choose  one  of  these  axes  as  the  axis  of  comparison  and 
then  to  quickly  select  positions  along  the  other  two  axes  through  menus.  Due  to  recent  performance 
enhancements  (see  below)  LAS  system  response  remains  quick,  even  when  generating  the  multiple 
visualizations  required  in  the  Slide  Sorter. 
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Figure  5.  LAS  Slide  Sorter  output  showing  the  HYDAE  model  intercomparison 


ii)  Animations 

A  generalized  animation  capability  has  been  added  to  LAS,  essentially  making  it  possible  to  view  on- 
the-fly  animation  sequences  of  any  visualization  that  the  system  offers.  This  gives  the  user  full  control 
over  space-time  domain,  variables,  contour  levels,  color  palettes,  and  other  graphical  layout  options 
when  generating  animations.  Animations  may  be  generated  for  maps,  zonal  and  meridional  sections 
and  even  line  plots. 
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Figure  6.  LAS  Animation  from  the  Global  operational  outputs 


iii)  Outputs-to-Google-Earth® 

Through  visualization  on  Google  Earth®  HYCOM  global  model  outputs  can  be  made  accessible  in 
classroom  settings  and  to  other  general  audiences.  It  also  provides  a  quick  reference  for  casual 
discussion  among  scientists.  To  address  the  high  resolution  (high  data  volume)  of  HYCOM  1/12 
degree  global  outputs  LAS  has  implemented  a  sophisticated  interface  to  Google  Earth.  Based  upon  the 
current  zoom  level  at  which  the  Google  Earth  user  is  examining  the  HYCOM  fields  LAS  automatically 
decimates  (“strides”)  the  data  that  it  accesses.  The  result  is  that  as  the  user  zooms  in  the  apparent 
resolution  of  the  data  continually  increases,  while  always  maintaining  satisfactory  performance. 


6 


Google 


^r1 


Figure  7.  LAS  output  mapped  to  Google  Earth® 


iv)  GIS-compatibility 

Like  outputs  to  Google  Earth  the  ability  to  output  to  the  emerging  open  standards  of  the  GIS 
community  is  an  important  outreach  tool  for  the  HYCOM  consortium.  LAS  has  added  a  beta-level 
capability  to  output  to  two  of  the  most  common  standards  that  are  being  advanced  by  the  Open 
Geospatial  Consortium  (OGC).  These  are  the  Web  Mapping  Service  (WMS),  which  makes  images 
(maps)  of  HYCOM  outputs  available,  and  the  Web  Coverage  Service  (WCS),  which  makes  binary 
grids  of  data  available. 

v)  Improvements  to  the  LAS  code  framework 

A  great  number  of  improvements  have  been  made  to  the  code  framework  that  supports  HYCOM 
model  dissemination  through  LAS.  Key  among  these  are  automated  decimation  (striding),  to  provide 
high  performance  access  to  high  resolution  fields,  when  viewed  over  broad  areas  where  the  full 
resolution  of  the  data  is  excessive;  server-side  analysis  through  OPeNDAP,  which  was  introduced  by 
building  upon  the  Unidata  THREDDS  Data  Server  (TDS  —  http://www.unidata.ucar.edu/projects/ 
THREDDS/);  and  new  tools  to  create  compatibility  between  curvilinear  and  rectilinear  grid 
representations  of  HYCOM  outputs. 

c)  HARDWARE 

Since  January  2007,  the  HYCOM  data  service  is  hosted  on  state-of-the-art  hardware  consisting  of  a 
Storage  Area  Network  (SAN)  that  links  6  RAID  arrays  providing  100  TB  of  total  storage  to  three 
servers  each  with  8  CPUS  and  32  GB  of  memory  via  a  high  speed  Fibre  Channel  network.  A  global 
file  system  which  provides  a  single  system  image  is  also  installed  in  the  servers.  The  high  CPU  count 
and  large  memory  machines  are  very  useful  for  multi-threaded  large  data  serving  applications 
operating  on  large  datasets. 
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RESULTS 


The  Hybrid  Coordinate  Ocean  Model  (HYCOM)  consortium’s  data  service  provides  access  to  datasets 
listed  below. 

•  1/12°  global  near-real-time  ocean  prediction  system  outputs  (updated  daily) 

•  1/12°  global  HYCOM+NCODA  hindcast  dataset  2004-2006 

•  1/12°  global  HYCOM  non-assimilative  simulations  2003-2006 

•  1/12°  Atlantic  Ocean  Prediction  Outputs 

•  1/12°  Pacific  Inter-annual  simulation  outputs  1979-2003 

•  1/3°  North  Atlantic  Simulation  1950-2004  (not  public;  used  internally) 

The  total  size  of  the  datasets  is  about  70  TB.  These  datasets  are  accessible  from 
http://www.hycom.org/dataserver  and  links  within  (see  screenshot  below). 


HYCOM  Consortium 
data  products  &  services 


DataServer  Home 

THREDDS  Catalog 

HYCOM-LAS 

FTP  Server 


Welcome  to  the  HYCOM  Data  Service  Website 


This  website  provides  access  to  near  real  time  global  HYCOM  +  NCODA  based  ocean  prediction  system 
output.  The  ocean  prediction  system  runs  daily  at  1800  hrs  and  produces  a  five  day  hindcast  (analysis)  and 
a  five  day  forecast.  These  data  are  typically  available  the  next  day  from  this  website.  This  website  also 
provides  access  to  several  other  global  and  regional  datasets  -  see  below.  The  data  are  in  NetCDF  format 
and  can  be  accessed  using  OPeNDAP  enabled  clients  such  as  Ferret.  NCO  etc.  A  Live  Access  Server 
I'LAS'iis  also  available  which  can  be  used  for  quick  online  visualization  and  downloads  of  data  in  several 
formats  (see  LAS  home  page  for  more  information).  OPeNDAP  URL's  for  all  these  datasets  are  available 


HYCOM  Global  1/12  SSH  snapshot 
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Quick  Links  to  HYCOM  Data 


Documentation 

OPeNDAP  URLs 

HYCOM  +  NCODA  Global  Near  Real  Time  Outputs  (APR  2007  -  present) 

http:  hvcom-coaps.fsu.edu  thredds  dodsC  db  nrt  analysis 

HYCOM  +  NCODA  Global  Hindcast  (Analysis:  NOV-2003  to  DEC-2004) 

http:  hycom.coaps.fsu.edu  thredds  dodsC  db  analysis 

Non- Assimilative  HYCOM  Global  Simulation  (2003  to  2005) 

http:  hycom.coaps.fsu.edu  thredds  dodsC  db  simulation 

North  &  Equatorial  Atlantic  Ocean  Prediction  System  (June  2003  to  Present) 

http:  hycom.coaps.fsu.edu  thredds  dodsC  atl  ops 

Interannual  North  &  Equatorial  Pacific  Ocean  Basin  Simulation  -  (monthly  means:  1979-2003) 

http:  hvcom.coaps.fsu.edu  thredds  dodsC  pacific  ecmvvf  simulation 

Disclaimer:  This  is  a  demonstration  product  from  the  HYCOM  Consortium  and  is  provided  as  is.  HYCOM  Consortium  does  not  warrant  or  suggest  that  this  data  is  fit  for  any  particular 
purpose.  Further,  neither  COAPS  nor  HYCOM  consortium  guarantee  av  ailability',  service  updates  or  timely  data  delivery 

07/02/2007  HYCOM  Consortium  COAPS.  FSU 

_ _ Zj 

I  I  I  I  [  I  I  |®  Mamet  |«U00%  -  ^ 

Figure  8.  Screenshot  of  the  HYCOM  data  service  website  listing  quick  links  data. 
IMPACT/APPLICATIONS 

This  system  provides  easy  and  efficient  availability  of  ocean  model  data  to  satisfy  ocean  data  needs  of 
researchers  and  operational  modeling  sites. 
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TRANSITIONS 


a)  Online  model  output  used  as  input  for  coastal  ocean  models 

b)  Online  model  output  used  as  input  for  biochemical  studies 

c)  Online  ocean  prediction  system  output  used  in  adaptive  sampling  of  the  ocean  by  sea-going 
physical  oceanographers. 
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